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ABSTSACT 

his publication provides some guidance to alcohol 
and drug education program administrators by clarifying the different 
levels of evaluation and the kinds of learning that can occur at each 
level. While it outlines the components and considerations for 
evaluation, it does not define a step-by-step procedure. In short, it 
serves as a diet rather than a recipe • A second objective for the 
publication is to talce away the threatening overtones to the word 
"evaluation" by showing the benefits to be gained and by pointing out 
the value that negative findings have for the ultimate success of a 
program. This book contains four sections: Why Evaluate?; Guidelines 
for Impact Evaluation; Suggestions for Process Evaluation; and 
Evaluation of Drug Programs (synopses of some examples of evaluations 
of alcohol and drug education programs) . The section of the book 
dealing with impact evaluation was developed by a task force 
sponsored by the Southern Eegional Education Board project. Enhancing 
Drug Education in the South. The task force included State-level 
program directors, representatives from Federal agencies concerned 
with alcohol and drug education, and people with expertise in program 
evaluation. (Author) 
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FOREWORD 



This publication marks one of the first attempts by the Prevention Branch, Division of 
Resource Development, National Institute on Drug Abuse, to share Information gleaned from 
projects sponsored by demonstration grants. There is no question that the schools are 
second only to families in their potential for influencing the drug-taking behavior of the 
young. In this report, the Southern Regional Education Board, an educational compact of 
14 Southern States, has crystallized some of the best thinking to date about the school's 
role and responsibility in the area of drug abuse prevention. This is one of a continuing 
series of tools that the Prevention Branch is making available to assist schools In their 
Important role in influencing young people. While this report is of necessity limited, it is 
anticipated that information will be added as new developments warrant and materials 
emerge to advance the state of the art of drug prevention and education. 

John R. Olsen, Ph.D. 
Chief 

Drug Abuse Prevention Branch 
Division of Resource Development 
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INTRODUCTION 



This publication provides some guidance to alcohol and drug education program adminis- 
trators by clarifying the different levels of evaluation and the kinds of learning that can 
occur at each level. While it outlines the components and considerations for evaluation, it 
does not define a step-by-step procedure. In short, it serves as a diet rather than a recipe, 
A second objective for the publication is to tal<e away the threatening overtones to the word 
"evaluation'^ by showing the benefits to be gained and by pointing out the value that nega- 
tive findings have for the ultimate success of a program. 

This book contains four sections: Why Evaluate?; Guidelines for Impact Evaluation; 
Suggestions for Process Evaluation; and Evaluation of Drug Programs (synopses of some 
examples of evaluations of alcohol and drug education programs). 

The section of the book dealing with impact evaluation was developed by a task force 
sponsored by the Southern Regional Education Board project, Enhancing Drug Education 
in the South.^ The task force included State-level program directors, representatives, from 
Federal agencies concerned with alcohol and drug education, and people with expertise in 
program evaluation. 

Xenia R. Wiggins 
Project Director 
Enhancing Drug Education 
in the South 



'The Project was designed to bring together State-level alcohol and drug educators from the SREB region 
in problem-solving conferences and task-oriented workshops. The 14 SREB States are Alabama, Arkansas, Florida, 
Georgia, Kentucky, Louisiana, Maryland, Mississippi, North Carolina, South Carolina, Tennessee, Texas, Vir- 
ginia, and West Virginia. 
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WHY EVALUATE? 



Many drug education programs have been ''crash" programs, created by public panic and 
pressure on public officials to "do something quick" about the drug problem. In response 
to what was defined as a crisis situation, we hurried to develop informational programs under 
the assumption that information would change behavior. In our rush we frequently began 
with inconsistent objectives and poorly thought out plans for monitoring, the results. Where 
evaluation did occur, it indicated that informational programs were not effective, even where 
information was objective and scare techniques were avoided. Consequently, drug education 
is being called to task. To receive continued support, program directors must offer new and 
clear approaches to drug education; they must demonstrate to their funding agency's 
satisfaction— and ultimately to the public's satisfaction— that these new directions do 
contribute to a decreased drug problem. 

In most cases program directors are short in evaluation skills. Their *'we have to do 
something now" response to pressure for accountability leads to a variety of activities, from 
highly sophisticated research to simple activity counts, all being submitted under the label 
of ''evaluation." Too often, the variety of approaches to evaluation that lies between these 
two extremes is not considered — the strengths and limitations of different levels of com- 
plexity in evaluation are not explored to help program directors determine the level thai best 
meets their program needs and budget. And too often, the reasons for evaluation and the 
importance of evaluation — beyond program justification — are not realized. 

WHAT IS EVALUATION? (Conceptual Issues and Definitions) 

The range of viewpoints on the reasons for and methods of evaluation varies greatly 
among program administrators. Some think that "good" evaluation requires formal research 
procedures, while others use personal opinions and subjective judgments. Although personal 
opinions may not be accurate evaluations, on the other hand it is not necessary to conduct 
highly complex, sophisticated research with rigid controls and precise measurement in 
order to gain reliable information for evaluating program effectiveness. 

While evaluation often uses research techniques for data collection and analysis, there 
are differences in the objectives of evaluation and research; consequently, the scientific 
design and the degree of scientific precision required are different. Evaluation is conducted 
to provide reliable information to program directors as a basis for decisionmaking. Constant 
feedback of findings into the program is needed so that changes for the good of the program 
and its clients can be made as soon as possible. It is not usually necessary to have the tight 
controls on data collection that are required for the sophisticated statistical analyses often 
employed by research. This does not imply that evaluation can be sloppy, but rather suggests 
that it should be more concerned with accurately reflecting trends and changes (as 
influenced by the program) in the target population, rather than developing new research 
tools and testing theoretical hypotheses. 

' While the point intended here is to have administrators recognize that evaluation need 
not be extremely costly and complicated, we do not want to imply that program directors 
should go to the other extreme. Personal judgments and simple number counts of activities 
sponsored, material distributed, people involved, etc., do have their place in program 
evaluation, but their limitations should be recognized. 
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It is important to clarify two different dimensions of evaluation. One Is "Impact 
evaluation," or a measure of changes that took place in the target population. The other Is 
"process evaluation/' a description of the target population and the techniques, methods, 
etc., which were used. For example, a drug educator needs to know how many and what 
kinds of people are being reached by his program In order to determine If there are 
segments of the population who should be involved but are not currently Involved (process 
evaluation). Simple logs and number counts will give him this kind of Information but will 
not tell him anything about the effect of his program on the audience's drug use decisions 
and behavior (Impact evaluation). Each type of evaluation has Its contributions and Its 
limitations, and administrators need to understand these In order to decide what level of 
evaluation Is needed to provide them the Information they need. 

Cost is a major factor In deciding the complexity of evaluation. The more sophisticated 
the tools and the more complex the evaluation design, the more costly evaluation will be. 
Considering the objectives of evaluation we have discussed, there comes a point where 
doing more sophisticated evaluation does not sufficiently Increase the reliability and validity 
of the results to make It worth the added costs. Later In this publication, some techniques for 
reducing costs are listed. 
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REASONS FOR AND AGAINST 

An honest answer to "why do we do evaluation?" Is very often "to justify the continued 
existence of the program." In too many cases, our primary aim Is to prove that a job has 
been done well In order to secure more funding, protect jobs, etc. The possibility of 
negative findings, and hence the possible loss of support, makes evaluation a threat. 

Even where findings are favorable, evaluation cannot guarantee continued funding, for 
there are often extraneous political forces that operate on decisions to continued or 
discontinue programs. Also, the time required for meaningful evaluation is often Incom- 
patible with funding practices. Funding agencies may want to know points of program 
impact that cannot be known until long after the refunding decision has been made. 
Consequently, the "evaluation" that many programs send to funding agencies seldom 
qualifies as "good evaluation." Since evaluation based on a funding motive cannot guaran- 
tee funding and does not always produce useful Information, administrators would do well 
to explore other reasons for conducting evaluation and benefits to be gained. 

The task force explored a number of reasons which support evaluation, along with 
reasons frequently given against evaluation (commonly referred to as the yes, but . . . 
syndrome). They are categorized as follows: 

Why Evaluate? Why Not To Evaluate? 

Humanistic Reasons 

1. Helps you know If you are meeting 1. You can't quantify people 
target needs 

2. Money spent on services may be wasted 2. Money spent on evaluation means less 
if you have no way to know if you are money spent on services. 

doing what you intended to do. 

Administrative Reasons 

1. You need to demonstrate that what you 1. No money 
are doing are steps to what you hope to 2. No staff 
accomplish. 

2. Evaluation is important for future 3. No time 
program planning — knowing what works 

and what doesn't work or, all else being 
equal, which approach Is cheaper. 

2 
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Sophisticated Reasons 
1. You can measure well-defined objectives. 1. Program evaluation "can't be done." 
it may be too early to evaluate program You can't isolate the influence of your 
success in terms of long-range goals, but program from the influence of other 
you can demonstrate that immediate social forces in operation at the time, 
objectives are in the direction of long- 
range goals. ' '2 

If the purpose of evaluation is to develop a better program, then negative findings are 
significant and useful rather than threatening. The program may learn its weaknesses, but it 
also learns its strengths. It has an advantage that it did not have before in knowing what 
areas need revision and more concentrated attention that will further the probability of a 
successful program. New strategies can be tried, evaluated, revised, etc., in a continued 
learning process. However, if the only objective of evaluation is to meet funding guidelines, 
there is not enough time in a funding period for this valuable learning experience to occur. 
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GUIDELINES FOR IMPACT EVALUATION 



Impact evaluation cannot show to. what extent a single program is responsible for bringing 
about a specific change because it cannot separate program influence from the influences 
of other current trends. However, it does demonstrate whether or not the expected change 
has occurred. Discovering that a desired change is not taking place under existing 
strategies is a significant finding. This has certainly been the case in drug education where 
evaluation has shown that informational approaches not only do not reduce drug use but 
may stimulate drug use or alternative drug use. Without such studies, drug education might 
have continued on its earlier path at great expense and waste of time. If there had been 
more evaluation of alcohoi education and anti-smoking campaigns, which used the same 
strategies that drug education has used, and if the findings had been widely distributed, 
perhaps drug educators would not have had to rediscover the fact that information has little 
influence in personal decision to use or not use drugs where emotional and social pressures 
are high. 

IN-HOUSE VS. INDEPENDENT EVALUATION 

An important early decision for prograrr; directors is whether or not to make impact 
evaluation an in-house function of the program. To some degree, this decision is influenced 
by the extent of in-house capabilities and funding available. However, there are additional 
issues which should be considered. 

There are arguments for either choice, and the director faces a tough decision in deter- 
mining which ''trade-offs" he can accept. In-house evaluation is usually less expensive than 
independent evaluation. At the same time, it runs a greater risk of being biased. If the 
evaluator (or evaluation team) is a member of the program staff, he has an investment in 
seeing the program succeed, not only in the psychological sense of wanting to be part of a 
"successful" program, but especially in the practical sense of wanting to secure his job. 
One suggestion for reducing the risk of bias in in-house evaluation is to submit the 
evaluation for outside peer review. 

The program evaluator who is a member of the program staff should be relatively free 
from responsibility for program development. Although evaluation staff should maintain close 
contact with program development staff, it is generally best not to assign the two functions 
to the same person. 

When evaluation is independent of the program, there is still a need for minimum 
in-house capabilities to judge the quality of services being purchased and to effectively 
use the evaluation findings in program decisions. Someone on the staff should know about 
existing services — available from agencies in the same system, for example — that would 
help reduce the costs of evaluation. 

In short, whether evaluation is done in-house or independently, there are minimum 
In-house skills needed by program staff who will be working closely with evaluation staff. 

Program staff need to be familiar with evaluation procedures and rationale to the extent 
that they can: 

1. Understand and use evaluation jargon 

2. Work with evaluation staff in translating program objectives into measurable terms 
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3. Make informed decisirns concerning the level of sophistication in evaluation design 
needed to get reliable answers to questions asked 

4. Judge the competence of evaluation staff and the cost of the total evaluation package 

5. Use the evaluation findings for program change 

PLANNING PROGRAM OBJECTIVES/EFFECTIVE USE OF A CONSULTANT 

The evaluation person, whether on program staff or independent, should serve as an 
"alter ego" to the program director. He should be free to make statements such as "You ^ 
can't do that in 2 days. Change your objectives or change your timespan." And ^'You can't 
measure mental health; you can measure divorce rate, suicide rate, admissions to mental 
health centers, etc., but these measures are not pure measures of mental health." He should 
also be free to suggest new questions that might be considered for evaluative research. 
For example, evaluation of drug education programs has consistently looked at changes in 
drug knowledge, attitudes, and use rates. This is old information now, and spending more 
time and money looking at the same questions would probably be a waste. Of course, the 
questions asked by evaluation tie in closely to the objectives of the drug education 
program. In the past, drug education has aspired to increase knowledge about drugs on 
the assumption that more knowledge will result in reduced use. Drug education is changing 
now in the direction of objectives for total behavior, not just drug-using behavior, and 
evaluative questions will be different. 

If we are going to base our "drug education" programs on the belief that drug-usmg 
behavior cannot be separated from other behavior, we should be looking at total lifestyles 
and learning more about the behavior patterns of people who do not become seriously 
involved in drug use. Rather than looking again at drug knowledge and attitudes, there are 
other questions concerning individual behavior which would give us better information 
about the success of programs in drug abuse prevention: 

1. What is the strength of interpersonal and intrapersonal skills? 

2. What is the capacity for responsibility and independence? 

3. What is the extent of neurotic maladjustment? 

4. What levels of anxiety exist? 

5. How well can the individual internalize and manifest behavior control? 

6. What is the extent of alienation? 

Whenever possible, the evaluation consultant should be involved from the beginning of 
the program when objectives are being defined. In this way, objectives can be specified 
according to measures to be employed, the information needed can be clarified, and time 
schedules can be established to insure the most appropriate times possible for data collec- 
tion. Evaluation designed and conducted late in the program operation will not give com- 
prehensive or valid information, nor will it allow for feedback into the program at different 
stages while there is still time to correct problems. 

A program director may choose not to evaluate every aspect of his program; he may 
have objectives that are immeasurable but important to the program because of the sense 
of direction they provide. Any objective to be. considered for evaluation must pass two tests: 
It must be feasible and it must be measurable. 

The task force outlined four essential components to include in defining objectives 

for evaluation; 

1. It must contain a description of the target population (e.g., all Washington high school 
students, or a random sample of 30 students from sophomore health education classes 
at Washington High), and the desired outcome to be measured. 

2. It must contain a statement of time involved or the ^^deadline" by which the desired 
outcome should occur. The time limit should be consistent with the objective (i.e., don't 
set a 3-month deadline for an outcome that can't possibly occur in that short period) 
and should vary for objectives set for different stages in the program. 
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3. It must contain an explanation of "key factors'' and ''key indicators." For example, 
alcohol education may have as its objective to reduce social problems related to 
drinking alcohol. One key factor might be alcohol-related automobile proble-T^s. Key indi- 
cators under this would be rates of arrest for driving under the influence, ,»nd alcohol- 
related automobile accidents. Another key' factor could be alcohol-related employment 
problems. Key indicators would be absenteeism resulting from alcoholism and industrial 
accidents caused by drinking, 

4. It must have a rationale which explains the logic underlying the choice of the objective 
as a step toward the overall goal. This will help avoid a complete change of objectives 
in mid-program. 

Objectives should be consistent with the needs of the community for which the program 
IS designed. Program directors will be more successful in defining objectives that respond 
to the community if they spend time with members of the target population, involve them in 
program planning and talk with people from other agencies concerned with the drug 
problem. It is also important that new program directors take into consideration the 
viewpoints for which some agencies know the drug problem and recognize that their view- 
point on community needs may represent only a part of the total picture. 

The evaluation consultant should anticipate negative findings and be able to work 
with the program director in planning alternative strategies and in understanding the 
reasons for the findings. At the same time, the program director should let the evaluator 
know in the beginning what his degrees of freedom are; that is, how much programmatic 
change can be tolerated within the real constraints of the system. 

REDUCING THE COST OF EVALUATION 

Most program directors would like to have more information about their program's 
impact, but they do not have the funds or staff to spare. Staff may keep simple logs of 
program activities, participant responses, etc., that provide valuably information at little cost, 
but these do not qualify as impact evaluation. There are some ways, however, to conduct 
more sophisticated program evaluation and still keep costs at a minimum. A number of com- 
ponents of evaluation costs and suggestions for reducing them are outlined as follows: 

1. Consultation: Evaluation expertise may not exist on program staff, but there is a good 
chance that it exists somewhere within the system of which the program is a part. For 
example, school systems often have research units. If it is necessary to go outside the 
system, faculty at nearby college or university and, particularly, doctoral candidates in 
need of a dissertation may be willing to design and write the evaluation in exchange for 
publication rights. The program director should recognize, however, that the faculty 
person has his own agenda and should take measures to insure that it does not interfere 
with evaluation needs. 

2. instrument design, dupiication, and testing: It may be possible to get help from research 
industries in designing and testing ihstruments. The program director gets the tool he 
needs and aids the research industry in developing a marketable instrument by providing 
a test sample in return. Again, college based research programs may lend a hand, or 
State psychological organizations may have a free services program. 

3. Administering questionnaires/ interviews: One good place to look for volunteer help is in 
social science classes at local colleges. Students are looking for an opportunity to test 
some of the skills they are learning in class and may need such experiences as part of a 
course requirement. Another source of help is volunteer organizations. Groups such as 
Junior League often have members with the expertise needed and with a desire to work 
with a social service agency. A number of drug education prograrrs have called on such 
volunteer groups for help in a variety of areas. Short-term training can produce a group 
of very effective workers. 

O 6 
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4. Coding instruments/ keypunching: Getting raw data into a usable form is one of the most 
costly components of evaluation. Again, students learning the skills needed and volunteer 
groups are good resources. High'tschool students could also do this aspect of a special 
science or drug education program. 

5. Computer time: Most computer centers have canned programs that are readily adaptable 
to the data. Computer time may be obtained from universities at little or no cost. Also, 
businesses in the community may be willing to donate computer time and programming 
help as a community service. It is important to know in the beginning what computer 
will be used and basically the kind of program needed so that the data can be coded 
and prepared (put on cards or on tape) in a manner compatible with the computer system 
and the program. 

6. Nonreactive measures: Nonreactive measures or unobtrusive measures are those made 
on naturally occurring events (e.g., sale of drug paraphernalia, school dropout rates) 
and therefore can be obtained from existing records without interacting with the popu- 
lation under study. These measures are generally less costly since it is merely a matter 
of collecting existing information or counting observed behaviors. Nonreactive or 
unobtrusive measures can also be made on events which occur naturally, but for which 
no records are kept. For example, one indicator of the extent of use of alcohol and drugs 
can be obtained by counting liquor bottles, beer cans, etc., found in garbage, and by 
monitoring the number of drug paraphernalia (syringes, etc.) found in the area. A person 
using such unobtrusive measures should be very aware of all the possible conditions 
influencing his "findings," so that he will not jump to wrong conclusions. A nonreactive 
measure needs to meet the following criteria: 

a. Variable — the frequency of the event must vary. For example, school dropout rates 

are not constant but, rather, fluctuate over time, 
b Feasible 

c. Reliable— the measure consistently reflects the same outcome in repeated studies 

d. Face validity— logically relates to behavior under study 

e. Ethical— since subjects are uninformed, their confidentiality should not be violated 
through reporting and identifying individual behavior. Nor should they be studied 
under potentially embarrassing circumstance (e.g., no one-way mirrors in restrooms). 

It is also possible to reduce costs by using available data if it provides the information 
needed and covers a time period close enough to the ideal time period to make it useful. 
Very often the same population (e.g., students in public school system) is inundated with 
questionnaires asking basically the same questions. Much time and money could be saved if 
agencies would coordinate their data collection or would borrow, when possible, from the 
information already collected by other programs. 

This discussion does not cover all the components of evaluation costs nor the ways in 
which costs can be reduced. A resourceful administrator with a little imagination can get 
an evaluation of his program with little additional costs and probably no extra staff. 

If the program director has evaluation funds and chooses to contract for services, he 
should ask for proposals with budgets so that he will know what he wants and how to 
recognize it when he gets it. 

USE OF FINDINGS 

The use to which evaluation results can or will be put depends on: (1) the point during 
program operation at which results are available; (2) the constraints in operation that may 
hinder making changes recommended by the evaluation results; and (3) the decisions of the 
program director and his consultants. 

The point at which evaluation results are available to program directors is an important 
factor in determining the use of findings. To some extent this point is determined by the 
length of the drug education program. In very short programs (even a program lasting 
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several months may be too short), there is not enough time available to evaluate, make • 
suggested program revisions, and reevaluate. In such cases, program directors are limited 
to feedback which occurs only after the program has ended/Ongoing programs or programs 
of long duration have the option to provide for periodic evaluative feedback. Some would 
argue that periodic feedback destroys the opportunity to measure the effectiveness of the 
original program by causing continuous program revision. The point to remember is that an 
objective of evaluation, whenever possible, is to provide a good information basis from which 
to make programmatic decisions and improvements. If evaluation results are available only 
at the end of the program, they cannot serve this function except to guide new programs 
which may be trying to accomplish similar objectives. 

When evaluation is built into every step of program development and feedback is 
continuous, the program director has access to findings at any point and can^usl them as a 
basis for deciding whether or not the next step should be carried'out as first planned. 
Continuous evaluation can serve an ongoing admint^atWe-management function. ^ 

The program director needs to recognize that continuous feedbadl^4l>m ev:aluation may, 
at some point, call for a program reyision including new objectives that'^would reqtJire— 
reevaluation and a need to collect new kinds of data. 

Evaluation findings would have greater value (beydmj their use to the individual 
program) if the final report and perhaps even significant periodic reports were shared with 
program directors or program planners who could benefit from the learning experience. Dis- 
tributing the information through publications or newsletters generally does not work unless 
it is accompanied by a planned followup which encourages the recipient to read the 
material before filing it away. The task force suggested bringing program planners together 
in a workshop to go over what has been learned through evaluation and how it can be 
applied to participants' programs. Such a workshop should be closely followed by technical 
assistance to help program directors implement new strategies, etc., suggested by evaluation. 

COST-EFFECTIVENESS EVALUATION 

Impact evaluation which does not include cost effectiveness leaves out a major 
component necessary to future program planning. While it is importaht to learn how close 
a program has come to reaching its goal, it is equally important to know the cost of getting 
there. If the cost of duplicating or continuing a successful program is prohibitive, the fact 
that the program was successful is of little consequence. Cost effectiveness is of particular 
importance to funding agencies. If two programs are equally effective, the funding agency 
will naturally choose to support the program which operates most efficiently. The objective 
of funding agencies is to cut down program costs as much as possible without seriously 
sacrificing program effectiveness. 

The general guidelines listed below offer suggestions for planning cost-effectiveness 
evaluation:^ 

1. Perhaps the most important point (and the first step) in planning cost-effectiveness 
evaluation is deciding the unit of service or output to be measured. Is cost to be 
rneasured "per student," "per student hour," or by some other unit of measure? 
("Student" is used here to refer to a member of the target population. It is not restricted 
to "student" in the traditional sense of the word.) Reporting cost for the total program is 
meaningless because it does not allow comparability between programs, which is a major 
reason for cost-effectiveness evaluation. Nor does it indicate what the cost covers. The 
primary criterion for selecting the unit of service is that it be measurable. 

2. Another method which allows comparability between programs is computing cost by 
program function (e.g., administration, teaching, public information, consultation to other 
programs, etc.). In this way, any services (e.g., clearinghouse function or public infor- 

. ^ These suggestions are adapted from "Cost Analysis System for Day Care Services," Soutlieastern Day Care 
Bulletin No. 3, Southern Regional Education Board, September 1971. 
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mation) offered by one program but not offered by another can be factored out in 
comparing the costs of the two programs. All expenses, including personnel, are listed 
by funbtion. Where expenses apply to more than one function, they are apportioned 
according to the best estimate possible. For-example, staff members such as the director 
will have responsibility in more than one program area. In this case their salaries are 
distributed under each function according to the proportion of time they estimate . 
spending in that function. 

Donated costs (goods and services) should be computed and included to give an 
accurate estimate of the real cost of duplicating the program. Costs should be computed 
only for the donated (or reduced costs) goods and services that the program would have 
purchased otherwise. Some resources may be free of charge but would not be computed 
as donated because they are readily available to members of the community and are 
not provided at a special time or place for the drug education program. 
An important decision in cost analysis is deciding the time interval for which costs are 
computed. For short-term drug education programs, this decision is not necessary since 
costs would be figured from program start to program end. For continuing programs, it 
is important to choose a time interval that coincides as closely as possible with affiliated 
agencies which may be involved in the bool^l^eeping process. The same int*erval should 
be followed for the succeeding periods of cost accounting. 

It is necessary to add the cost of equipment purchased to obtain the total cost of the 
drug education program. The major reason for malting a distinction between supplies and 
equipment is to allow for depreciation of equipment. Separating equipment cost from 
supply cost permits differentiating between costs that include the full price of equip- 
ment (the out-of-pocltet actual expenditures) and the costs that include only a yearly 
depreciation allowance for equipment (the true cost). The "Internal Revenue Service 
Depreciation Guidelines and Rules" is helpful in determining depreciating rates for 
various Mods of equipment. Including major renovations to buildings. 
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SUGGESTIONS FOR PROCESS EVALUATION 



In the previous section we were concerned primarily with impact evaluation or measuring 
,,the effect of an alcohol or drug education program. This section concerns process evaluation 
or "What was done to whom" to produce what effect. There are some simple measures 
that help program directors monitor program progress and do not require much expense or 
staff time. An important objective of such records is to keep track of what is going on in the 
community (critical incidents) that might affect program results. 

DAILY LOGS 

Alcohol and drug education program directors often ask their staff to keep daily logs of 
activities, regardless of how trivial they might seem at the time. The logs usually contain 
descriptions of activities, people contacted, their reactions^ impressions, etc. Such descrip- 
tions can help staff recognize obstacles to program implementation, monitor progress in 
winning support from other community agencies, and monitor problem resolution. They 
provide insight into changes in institutional structures, give a picture of attitudes of people 
in key postions and also indicate the amount of staff time required for different program 
activities. The following is an example of a log entry and the kinds of problems it can 
highlight: 

"August 25 — Escorted by high school principal, visited County Health Department, 
Community Development Center, hospital. President of School, State Senator. Being 
introduced in this way will prove, I think, to be a distinct advantage. Evidently the 
principal is a respected citizen and leader in the community. Talked with Episcopal 
rector. Very status conscious. Pictures alcohol attitudes in (community) very narrow. 
Felt that a major obstacle for me would be students wanting to know whether I drink. 
Said he thought I should realize that I was going to lose effectiveness with one group 
or another." 

In addition to daily logs, some program directors may ask their staff to submit written 
annual or quarterly reports covering activities, impressions, evaluations, and suggestions. 

EVALUATION CONFERENCES 

Program directors can learn a good deal about problems their staffs are facing and their 
staffs' success in resoivrfg them through periodic staff meetings. Some alcohol and drug 
education programs have made effective use of year-end evaluation conferences. Local 
citizens and outside consultants are invited to react to the program's activities. Often, new 
directions and emphases will emerge from such conferences. They provide an opportunity 
for input from the population being served. 

DESCRIPTIVE EVALUATIONS 

There are a number of simple records that describe program activities, who was 
involved, etc. These include: 

1. Records of the activities sponsored by the program and the number and categories of 
people involved in each. This helps show where program priorities lie and where gaps 
exist in the population for which the program was designed. 
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2. Records of staff time spent in initiating and planning activities. This indicates to the 
program director how effectively his staff is being utilized and what proportion of time is 
devoted to various program functions. If the program has planned times for staff 
functions to change (e.g., an objective of the program may be to train people in the 
community to take over and continue the program so that staff functions would appropri- 
ately change from "drug educator" to "consultant."), such records will indicate whether 
or not that change is occurring. 

3. General information requests (requests for information about the program) 

4. Requests for publications and other material 

5. Records of speaking engagements, meetings attended, etc. 

6. Records of media coverage of the program 

7. Records of requests for consultations (with continued updates as to what has occurred) 

All of the above simple measures help to show the program's visibility. 

CONTENT EVALUATION 

It is a good idea to evaluate workshops and other educational efforts sponsored by the 
program. In addition to descriptive information mentioned earlier, program directors may 
want to measure the impact of workshops upon participants through brief subjective 
assessments of topics, formats, instructors, teaching procedures, etc., which ask participants 
to identify positive and negative aspects. 

SPECIAL STUDIES OR SAMPLE STUDIES 

In contrast with ongoing evaluation, special studies are conducted for a limited amount 
of time. For example, program directors may ask their staff to keep very careful records 
of their time for a 1-week period. Of course, there is the possibility that setting aside a 
special time of study will influence the way in which staff spend their time and may mean 
that the results do not truly reflect the usual staff patterns. Another example is to closely 
monitor telephone calls, letters, etc., for a given period of time to study the kinds of requests 
for consultation, material, etc., that are being made of the program. 
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RECORD OF CONTACTS AND MEETINGS 



Individual recording this information 



Project name and State 



Date of 
contact 
or mtg. 



Group, organization, or 
In- Out- person calling meeting 

coming going or with whom contact 

was made 



Major purpose or 
subject of contact 
or meeting 
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EVALUATION OF DRUG PROGRAMS 



The descriptions or synopses following each study are not meant to be judgmental of the 
program; rather, they are impartial descriptions written for the benefit of persons planning 
evaluations or similar studies of drug use, attitudes, knowledge, etc. 

These studies were not chosen on the basis of whether they were considered to be 
"good" or "bad" approaches to evaluation. They were included because they were the most 
accessible. Among the many evaluation studies identified and requested by the Southern 
Regional Education Board (SREB) project, these are the ones that were available in 
published form or sent to the project in response to requests for copies. 

Exclusion of similar studies does not imply that the ones listed here are the only 
possible approaches to evaluation and measurement. 

Nor do we intend to imply that the approaches to drug education evaluated in the 
included studies are the best approaches to drug education. Most of these evaluations are 
designed for programs based on the information model, because this is what most of drug 
education has been for the past few years. Drug educators are beginning to look beyond 
information dissemination to personal skill development as a more effective approach. 
Evaluations of these newer programs will look very different. 

Capone, Thomas; McLaughlin, James; and Smith, Frederick. "Peer group leadership program 
in drug abuse prevention 1970-71, academic year." Journal of Drug Educatior), Vol. 3. No. 
3, Fall 1973. pp. 201-245. 

(The following is a direct quote of the "Summary and Conclusions" of the article, 

pp, 242-243) 

"The Peer Group Leadership Program in Drug Abuse Prevention developed as an 
attempt to utilize the impact students have upon each other in order to prevent add combat 
drug usage among high school students in New York City. In its initial year (1970-71) it 
extended to 16 high schools. In the planning phase, designated Phase I, student, teacher, and 
principal participants were brought together to plan for the development of subsequent 

phases of the program. 

"Phase II consisted of a five day training and orientation period. Intended participants 
included the principal, one selected teacher, and eight selected students from each school. 
The workshop was intended to foster in the participants skills which would sen/e them as 
change agents within their student bodies. Students were also to formulate plans of action 
at this time which could be implemented upon return to their respective schools. 

"The purpose of the present evaluation was to determine the extent to which the follow- 
ing program objectives, as stated in the program proposal, were met: 

1. To develop an educational program which has as its focus the preventive approach 
to the problems of teenage drug abuse. 

2. To enable students to develop for themselves the skills in the process of facilitating 
healthy peer group relationships. 

3. To help students become effective leaders and change agents among their peers. 

4. To enable program students to formulate objectives. 
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*The evaluation methodology included interviews with the Director of the Peer Group 
Leadership Program, the assistant to the Chancellor, and the Associate Director of the Bureau 
of Planning and Innovation. Questionnaires were administered to principals, Teacher- 
Facilitators, and Peer Group Leaders. In addition, the evaluators made several on-site visits 
and attended several training sessions in order to observe program operations firsthand. 

"On the basis of the findings, the following conclusions can be drawn regarding the 
extent to which the stated program objectives were met: 

1. An educational program was developed which had as its focus the preventive approach 
to the problems of teenage drug abuse. The nature and extent of program development 
at each school appeared to vary considerably. Approximately 17 percent of the Peer 
Group Leaders indicated their group had done nothing in this regard. However, most 
students appear to have enthusiastically engaged in this phase of the program. 

2. Several training sessions were held to enable students to develop skills in facilitating 
healthy peer group relationships. Hopefully the training sessions would lay the ground- 
work in the development of effective leaders and change agents among their peers. At 
present, it is too early to expect evidence of the extent to which such skills have been 
learned and successfully applied. 

3. It does not appear at each school, that students, or even teacher facilitators, have 
formulated a clear statement of plans of action. At present, as might be expected at this 
early stage, only general formulations of program objectives and activities have been 
made. Hopefully, movement will be made toward developing greater specificity. . . . 
Moreover, it would appear that specific activities should be developed with the aid of 
consultants. It is necessary to foster cooperative formulations of certain common goals 
among schools as well as between teacher-facilitators and students at a given school." 

Doerr, Dale; Kabat, Hugh F.; Sheffield, William J.; and Skinner, William J. "On the campus 
. . . drug abuse programs." Journal of the American Pharmaceutical Association, \/o\, 
NS7, No. 9, September 1967. 

University of Minnesota 

To stress student involvement, the American Pharmaceutical Association, student 
chapter, was given responsibility for planning, publicity, and programming. Sponsors planned 
. a 2-day afternoon conference. Attention was given to timing — both time of day and time in 
the academic quarter — to ensure student attendance. 

A film, "Narcotics, the Decision," was shown the first day with the main presentations, 
a series of talks followed by panel discussion to be given the second day. The main program 
featured seven half-hour talks followed by 1-hour panel discussions. 

Program sponsors evaluated the conference as follows: 

1. For maximum impact, the entire program should be presented in 1 day. This conclusion 
was based on the drop in attendance from the first day to the second day. 

2. The 1-day program should be no longer than 2V2 hours. It was felt that the individual 
presentations had been too long and that the four talks on use and abuse could have 
been given as one presentation. 

3. A followup program for residence units would carry the message to smaller groups in 
a more intimate setting. 

4. The program should be presented every other year. 

University of Texas 

The professional Pharmacy Fraternity sponsored a symposium during which five 
speakers each made 20-minute presentations. A question-and-answer period followed. The 
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program was conducted in the evening in the student union hall and reached an audience 
of 350-400 stude,nts. (The article describes publicity measures for the program.) 

Sponsors were disappointed in the turnout and concluded that a successful program 
. . depends on well-known speakers, a topic of vital interest, and heavy advertising." 

Butler University 

The professional Pharmacy Fraternity sponsored a series of group discussions in housing 
units climaxed by a formal symposium. The total program lasted 1 week. Each housing unit 
was visited by a three-man team from the Pharmacy Fraternity that led group discussions 
on drugs, their use, and misuse. The symposium speakers included a psychiatrist, pharm^ • 
cologist, research specialist, and inspector for the Indiana State Board of Health. The 
program received*both' newspaper and TV coverage prior to the symposium. Planners judged 
the program a success, based on near-capacity attendance. 

Johnson, Barbara B. '*A junior high school seminar on dangerous drugs." The Journal of 
School Health, Vol. 38, No. 2, February 1968. Pp. 83-87. 

Bancroft Junior High School in the San Laredo Unified School District, California, 
conducted a drug education program for eighth and ninth grades. The progr ' ^m objectives 
were to help each student establish a set of values, look at himself in his world, look at peers 
who are an Important part of his life, and determine what he would do when faced with 
decisions about drug use. 

Stage I: English classes were divided into six or seven discussion groups with a student 
leader and recorder. Leaders were given guidelines such as encourage all students to talk, 
and leaders kept records anonymous. Students reacted to seven statements about drugs, 
for example, "I feel I will never use drugs/' "I would like to know these things about drugs." 

Stage II: A panel of experienced persons responded to students' questions in two separate 
assemblies by grade level. 

Stage III: Film— "Narcotics— Pit of Despair" 

The evaluation is based on students' reaction to the program and its influence on their 
attitudes and on the subjective feelings of the program planners. The report does not indicate 
any measure of student attitudes before the programs. 

Evaluation In English Classes, — Sixty-five percent of the students responded favorably to 
discussions in English class. Eighty-three percent of eighth graders and 66 percent of ninth 
graders felt the seminar had added to their knowledge of drugs. Approximately the same 
percentage reported that the seminar had influenced them against making drugs a part of 
their lives. 

Evaluation of Panel — Students were disappointed by the panel because their questions were 
not answered, discussion was not related to the immediate environment, and panelist spoke 
either above or below the students. Suggestions for improvement include having students 
on the panel or having students interview "experts" either in front of their peers or to report 
back to their peers. 

Evaluation of Film — Evaluation reports that students were favorable to the film. 

Lewis, Jerry M.; Gossett, John T.; Phillips, Virginia Austin. "Evaluation of a drug prevention 
program." Hospital and Community Psychiatry, April 1972. 

In October 1969, a questionnaire surveying drug use was administered to all junior and 
senior high schools in the Dallas Independent Schoo] District. Two weeks before the district- 
wide survey, a 2-day drug abuse prevention program was held at one of the larger senior 
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high schools. The questionnaire to be used in the district-wide survey was administered 
three days before the program to a 10 percent random sample (315 students) at the school. 
A nearby control school, similar in characteristics to the experimental school, also con- 
ducted a random sample survey at the same time. Both schools were then resurveyed during 
the district-wide survey. 

The drug program in the experimental school was presented by a private organization. 
The organization trained 15 percent of the student body as group leaders to conduct small 
group meetings in which the entire student body participated. Comparing the results of the 
before-and-after questionnaires for the two schools shows that . . of the 32 drugs or groups 
of drugs, two reflected greater usage in reported usage in the school that held the preven- 
tion program than in the control school. In 11 categories both schools had identical rates of 
change in reported use, generally a 1 or 2 percent increase. In 19 categories the control 
school had a larger increase in reported use than did the school with the prevention program. 
Thus while the antrdrug program may have played a part in the very slight decline in usage 
in 19 categories of drugs, it failed to do so in the other 13. Chi-square analysis of this result 
indicates that the pattern of 19 'successes' and 13 'failures' does not differ significantly 
from chance." 

Mason, Michael L. Drug Education Effects, Final Report, Young Adult Service, Gainesville, 
Florida. 

National Center for Educational Research and Development (DHEWLOE) 

Grant No. OEG-4-71-0070 

(The synopsis below is quoted from ERIC catalogue, entry no. ED 071-011) 

"This is a research project that was intended to study the effects of a factual drug 
education program on the attitudes of high school and junior high students toward vhe use 
of psychoactive drugs. The approximately 250 eighth and twelfth grade students involved 
in the study filled out a number of questionnaires designed to measure a variety of their 
attitudes about psychoactive drugs immediately before and after participating in a drug 
education program that relied heavily on the presentation of known facts about a variety of 
drugs. Analysis of the data indicated that the students learned about the given drugs to a 
highly significant degree, their curiosity about the effects of "mind-expanding" drugs was 
increased, and they exhibited an increased tendency to deal with psychologicaJ discomfort 
through the use of drugs. At the same time they reacted more favorably toward the legaliza- 
tion of marijuana and a reduction of penalties for drug use, and less favorably toward 
present emphasis on a legal approach to the use of drugs. The primary conclusion of the 
study is that drug education is not an effective means of suppressing the use of drugs. 
References and numerous tables are included." 

Monk, Mary; Tayback, Mathew; and Gordon, Joseph. "Evaluation of an antismoking program 
among high school students." Program Evaluation in ttie Healtfi Fields. New York: 
Behavioral Publications, 1969. 

Two comparable senior high schools in Baltimore participated in an evaluation of an 
antismoking program. The objective of the program was to persuade students to stop or not 
to begin smoking. Prior to beginning the campaign, both schools completed the same 
questionnaire. Students in the experimental school had 26 exposures to the antismoking 
campaign over 7 months. There was no such activity in the control school. The program 
concentrated on the connection between lung cancer and smoking and consisted of the 
following basic activities: 

1. School-wide assemblies over the PA system involving principal, student council, 
commissioner of health, cancer research scientists and physicians. 



O 20 

ERIC '28 




2. Sixteen posters displayed in home rooms at semi-monthly intervals. 

3. Two letters from commissioner of health mailed to all students. 

4. Series of articles in school newspaper. 

5. Two leaflets distributed: ''Shall I Smoke?" (American Cancer Society), and 'Why Buy a 
Pack of Trouble?" displayed in gymnasium. 

In addition, the school physician and nurse held group discussions on smoking and 
health with nearly all 10th grade students in groups of six to eight boys. Teachers were 
asked to have class discussions on smoking and to initiate appropriate classroom projects. 

Findings 

In late spring, both schools again filled out the same questionnaire they had completed 
in the fall. Teachers in both schools completed forms indicating how many students had 
taken part in classroom discussions or projects on cigarette smoking. In addition, interviews 
were conducted with 95 students representing all grades and courses in the experimental 
schools. 

The questionnaire measured changes in rates of smoking and changes in attitudes 
toward smoking. The antismoking program did not appear to affect the total amount of 
smoking in the experimental school. Change in smoking for all grades in both schools was 
negligible. 

The article gives a detailed report of comparisons of the two schools in students' 
responses to eight attitude statements about smoking. The desired change at the experi- 
mental school was greater than at the control school for only two statements, and for only 
one of these (''Smoking is dangerous to health") was the difference between schools 
significant. 

Although the questionnaire did not show it, 22 of the 95 students interviewed stated they 
smoked less in the spring than in the fall (when carnpaign began); however, they were still • 
regular smokers. The objectives of the interview were to measure students' retention of 
information provided in the campaign and to measure students' reactions to the program. 
Considering the theme of the campaign (the relationship between cigarette smoking and lung 
cancer), only a small proportion (16 percent) of students mentioned this first as a way smok- 
ing affects health; an additional 5 percent reported "they" say smoking causes lung cancer. 
Fifty-three percent gave as their first response that smoking causes shortness of breath. 

The interview showed that the poster had been by far the most noticed part of the 
program. Slightly more than half remembered the PA system lectures and less than half 
recalled the leaflets or mentioned the questionnaires. 

Conclusions 

The article suggests several points to consider in interpreting the findings: 

1. A program of 7-months' duration may be too short to counteract prosmoking influences. 

2. Greater teacher and student participation may have increased program effectiveness. 

3. The program was conducted at a school where there were a large number of regular 
smokers who are more likely than nonsmokers to resist antismoking education programs. 

4. Other studies have shown that most students who smoke begin a smoking habit between 
ages of 12 to 15. Future programs should concentrate on elementary grades. 

5. The students who smoke the most are the most difficult to reach through regular school 
activities. 

Norwalk, Dorothy. "Innovations in drug education." Journal of School Health, Vol. 39, No. 4, 
April 1969. Pp. 236-239. 

A drug education program for Suitland Senior High School, Maryland, was planned by a 
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student committee composed of three students from each class. The stated objectives for 
the program were to provide accurate information and to provide a variety of viewpoints 
about drugs, drug use, drug users and their reactions to drugs. 

Teacher preparation consisted of viewing a film on LSD produced by the U. S. Navy, 
receiving a packet of materials, and hearing an illustrated talk by a health educator from 
the University of Maryland. 

Prior to the program, all students saw the LSD film in small groups. Some students saw 
additional films. 

The entire student body met for lectures by a leader in the drug field. Presentation 
Included introduction to psychological, physiological, sociological, and legal aspects of drug 
use and abuse. Following the lecture, students returned to homerooms for 1-hour discussion 
sessions with the teacher and resource people (doctors, pharmacists, nurses, professors, 
BNDD representatives, lawyers, and education directors of State and Federal drug programs). 

While half of the student body met with resource people, the remaining half viewed 
three films: **Hide and Seek," "Narcotics: Pit of Despair,'' and "Drugs and the Nervous 
System." Th^e next hour, the two groups reversed the activities. 

Evaluation 

Immediately following the program, students filled out a form provided by their teachers 
(1,044 students responded). 

Results: 75 percent rated overall value of program as excellent or very good, 18 per- 
cent — good, 5 percent — fair, 2 percent — poor 
Lecture — 47 percent rated excellent or very good. 
Films — 75 percent excellent or very good 
Discussion groups^ — 68 percent excellent or very good 

The teachers' overall value of the program: 70 percent rated excellent or very good, 28 
percent — good, 0 percent — Fair, 2 percent— poor 

The consultant's overall value of the program: 70 percent rated excellent or very good ^ 
. and 30 percent — good 

O'Keefe, M. Timothy. "The anti-smoking commercials: A study of television's impact on 
behavior^" Public Opinion Quarterly 35:242-248, Summer, 1971. 

Self-administering questionnaires were distributed to area junior and senior high schools 
and selected freshmen classes. A total of 621 responses was obtained. In addition, a general 
population survey of 300 area residents was conducted by telephone. The purpose of the 
survey was to determine the effectiveness of a television antismoking commercial in persuad- 
ing people to stop smoking and in preventing others from starting. Results of the survey show 
that commercials appear to have no effect on the majority of smokers who wanted to con- 
tinue. They merely provided support to those who wanted to stop. The greatest impact was 
with younger people. Smokers recognized the health hazards, but few intended to stop or 
cut down. 

O'Rourke, Thomas R., "Assessment of the effectiveness of the New York State drug curricu- 
lum guide with respect to drug knowledge." Journal of Drug Education, Vol. 3, No. 1, 
Spring 1973. Pp. 57-66. 

The purpose of the study was to assess the effectiveness of a program based on the 
revised New York State Curriculum Guide in increasing students' knowledge about drugs, 
alcohol, and tobacco. The underlying rationale of the program was that increased knowledge 
would assist in reducing the incidence of drug abuse. 

The study was conducted with a total of 446 students in four high schools in Nassau 
County, New York. The two experimental schools received the new program, and the control 
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schools received a regular health education program without special guidelines. There was 
no pretest given to any of the schools. Comparisons are made between the control and 
experimental schools on a knowledge test given to both groups after the program. The author 
used analysis of variance to measure significant degrees of difference in cognitive achieve- 
ment between the two sets of schools. 

The experimental group scored significantly higher in knowledge on each of the thr^e 
subtests (alcohol, knowledge, and tobacco) and on the entire test. When comparisons are 
made by sex of respondent, the program appears more effective for males than females. 

Pollock, Marion B. "An evaluation instrument to appraise knowledge and behavior regarding 
use of stimulants and depressants." The Research Quarterly, Vol. 39, No. 3. Pp. 662-7. 

"This article describes the process by which a two-part evaluation instrument was 
developed to assess knowledge and behavior regarding the use of stimulant and depressant 
substances. A trial instrument was administered to 72 college-bound high school seniors. 
Analysis of the responses was used to construct a refined second version then administered 
to 467 men and women at California State College at Long Beach. 

The article draws the following conclusions: 

"1. There was a demonstrated need for instruction, and for an instrument that could be used 
to reveal strengths and weaknesses in present programs. 

2. A test was constructed that would (a) evaluate high school graduates' knowledge about 
the use and effects of tobacco, alcohol, and dangerous drugs and narcotics, and 

(b) reveal present practices in use of these substances. 

3. Use of instructional objectives as the source of a representative pool of items was dem- 
onstrated to be a valid process for the construction of improved health education tests. 

4. The test satisfied logical and statistical criteria of validity, reliability, and item 
discrimination." 

(The above is quoted directly from the article.) 

Selvin, Julius B. "Effectiveness of programmed materials in teaching a secondary school 
health education unit." The Research Quarterly, Vol. 39, No. 3. Pp. 704-7. 

(The synopsis below is quoted directly from the article, page 704.) 

"This study was designed to determine if intrinsically programmed materials and 
techniques can be effectively used to teach a health education unit to secondary school 
students, and to compare levels of achievement resulting from the utilization of programmed 
materials as opposed to traditional classroom methods. Twelve secondary school health 
education classes in New York City, involving 343 students, were divided into matched 
groups. Branched, self-teaching materials on the topic of alcohol education were presented 
to the experimental group to be studied at home for a three-day period without recourse to 
classroom instruction. The control group was taught the topic in a 3-day unit utilizing tradi- 
tional classroom teaching techniques. The results of the achievement test administered at 
the conclusion of the unit indicate that branched or intrinsically programmed materials and 
techniques can be effectively used in the teaching of a secondary school health education 
unit and that in this particular investigation the experimental group learned significantly 
more than the control group." 

A study of Student Behavior Changes in the SPARK Program Intervention-Prevention 
Centers conducted by the New York City Board of Education SPARK Program for the 
Addiction Services Agency of the City of New York, April 1973. 

The SPARK Program is a drug prevention program that was conducted in nine New 
York City high schools with indicators of high need. Some of the program activities include 
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student-le.d peer group programs, identification and referral of drug abusers to treatment, 
classroom and assembly information-oriented programs, and teacher training. The primary 
activity in all schools was counseling — individual, rap sessions, ongoing group sessions — 
for students whose behavior patterns indicated a high potential for drug abuse. The study 
was conducted to test the hypothesis . . that participation in SPARK program counseling 
would alleviate negative behavior, anti-social behavior,.and low personal achievement which, 
the literature suggests and the SPARK program assumes, are highly correlated with drug 
abuse factors which tend to foster drug abuse.'' 

A random sample of 100 students was selected from the ongoing group counseling 
sessions in each of the nine schools during the September 1972-January 1973 semester 
(900 students, total). During this semester, these students' records were examined to monitor 
absentee rate, referral to dean for disciplinary purposes by classroom teachers, major 
subjects failed, overall grade point average and conduct ratings. The findings of this exami- 
nation were compared to records for the same students during the September 1971 to 
January 1972 semester, when they were not participating in the SPARK program. "In every 
category studied, the aggregate population of the sample . . . showed a-significant change 
in each of the five indicators selected." Statistical analysis for the total sample, using a 
two-tailed t-test, shows that all changes are significant. The study also shows findings for^ 
each individual school where the results vary. 

Swisher, John D., et al., 'Tour approaches to drug abuse prevention among college students." 
Journal of College Student Personnel, Vol.14, No. 3. Pp. 231-235. 

The purpose of this study was to compare the relative effectiveness of four small group 
approaches to drug abuse prevention among college students: 

1. Discussion group — discussion of issues and problems surrounding drug abuse on factual- 
informational basis. Group leaders were instructed not to let the group become feeling- 
oriented. 

2. Relationship counseling group — students were free to explore topic in any manner they 
chose. Counselor did not impose his opinions on group but served to help members 
understand and express themselves. 

3. Reinforcement counseling group — included counselor and two college-age role models 
who were knowledgable about drug culture but had never used drugs. They were to 
guide discussion toward reasons for not using drugs and to reinforce students' state- 
ments that supported their not becoming involved with drugs. 

4. Reinforcement counseling group — included counselor and two college-age role models 
who were reformed drug abusers. Counselor functioned in same manner as group 3. 

All of the students (approximately 341) in required health courses during winter term 
were randomly assigned to these four group types. The control group (N = 33) was taken 
from two other health classes offered during winter term in which drugs were not discussed. 

Evaluation criteria included gain in knowledge, change in attitudes, and reduction in 
drug abuse rates, both immediately following the study and in a 3-month followup. The three 
major instruments used were a 41-item scale covering various issues (e.g. legalization of 
marihuana) and a 35-item health habits scale to measure students' current involvement. 



1. All students gained in knowledge regardless of group type. Data did not indicate any 
differential impact by type of group. 

2. Students shifted their attitudes in a liberal direction. The scale measured three separate 
realms: attitudes toward personal use, attitudes toward drug abusers, and attitudes 
toward drug laws. It was impossible to tell if the shift toward liberalism was in all realms 
or perhaps in just one realm. 



Findings: 





3. There were no statistically significant changes in extent of reported drug use; however, 
two of the counseling approaches reported lower rates of drug use than the discussion 
and control groups. Three of the four experimental groups reduced their consumption of 
alcohol and cigarettes. 

4. The inclusion of models (i.e., drug experienced and non-drug-experienced) had no 
differential impact in this study. 

Swisher, John D., and Crawford, James L. *'An evaluation of a short-term drug education 
program." The School Counselor, March 1971. Pp. 265-272. 

The program took place in a private school and was set up and directed by a joint 
student-faculty committee. The three guidelines planned for this meeting were (a) The 
faculty and administration would not take part in the program, (b) a psychiatric point of 
view should be presented, and (c) rehabilitated drug users from a comparable socio- 
economic class should be involved. 

The program was given to four high school classes. Each class session varied slightly. 
The program was offered on four consecutive Thursday mornings. The ninth grade was set 
up in three 1-hour group sessions with the same psychiatrist conducting the session each 
week. The 10th and 11th grades had a 1-hour group session, conducted by the three 
psychiatrists and then split into three smaller groups for the remaining 2 hours. The session 
for the 12th graders was similar to that of the 10th and 11th, but involved several rehabilitated 
drug users. 

In order to evaluate the impact of these three approaches the following instruments 
were employed: 

1. A 14-item attitude scale that measured opinions about issues related to drug abuse, 

2. A 30-item knowledge scale that measured knowledge about drugs in five areas, including 
narcotics, marihuana, LSD, amphetamines, and barbiturates, 

3. A 33-item behavior scale which assessed the health habits of students and their motives 
for these habits, 

4. A 7-item student evaluation scale which was designed to assess student perceptions of 
this particular program. 

These scales were administered to the students participating in the drug program 2 
days before the first Thursday session and again 1 week after the last session. 

An analysis of variance for the attitude scores yielded no significant differences between 
or within treatments. The data from the attitude tests revealed that the mean scores for all 
of the grade levels changed only slightly in a pro-drug direction but was not a statistically 
significant shift. However, the analysis of variance for the knowledge scale yielded significant 
differences for the school as a whole on pre- and posttesting, as well as significant differ- 
ences among the various grade levels with the freshmen having the least amount of 
knowledge and the seniors possessing the greatest amount of information. 

In terms of the students' perceptions of the drug education effort, the seniors rated this 
program extremely high relative to other schoolwide programs. The sophomores and juniors 
who were exposed to lectures and then group discussions rated their program as being more 
informative than the other grade levels. The freshmen who were exposed to group discus- 
sions only reported more discussion with their parents. 

Swisher, John D., and Herman, Richard E. "Drug abuse prevention." The Journal of College 
Student Personnel, September 1970. Pp. 337-341. 

A conference entitled "A Retreat on the Hazards of Drug Abuse" was presented to the 
students and faculty of Temple University. The purposes of the conference were to inform 
students, faculty and administrators about drugs and to stimulate their interest so that they 
would further disseminate relevant information to the entire university population. 
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The program included group discussions led by a variety of experts (psychiatrists, 
pharmacologists). In order to evaluate the group, pre- and posttesting were administered 
seeking attitude changes and information gains. Also included was a followup questionnaire 
sent out 6 weeks later which focused on the participants' activities in terms of disseminating 
information to the university at large. A control group was also set up from the individuals 
who were invited but for some reason could not attend. Three evaluation Instruments were 
developed for this study. The evaluation instruments were: 

1. The Drug Abuse Scale which contained achievement items focused on knowledge about 
different types of drugs. The attitude subtest focused on issues related to drug abuse 
(e.g., legalization of LSD or marihuana). These items were randomly scattered among the 
objective items on the assumption that response sets would be less likely to occur. 

2. A Conference Evaluation Form that assessed the participants' impressions and reactions 
to their experiences at the retreat. 

3. The Drug Education Activities Scale which asked the participants to record their behavior 
following the conference. More than 85 percent of the participants and almost 100 percent 
of the control group returned this questionnaire. 

Results: 

''Gain in Knowledge'' 

The difference in knowledge gained was statistically significant, which indicated that 
attendance at this type of conference had an impact on the participants' general level of 
knowledge concerning drugs. There were no differences found when undergraduates, 
graduates, and staff were compared. 

"Attitude Changes" 

The attitude data collected as part of the evaluation revealed that at the outset the 
participants and controls generally had conservative attitudes with regard to the various 
issues. All of the statistically significant changes in attitudes occurred among the under- 
graduate students. Data are given in the article to further explain the exact attitude changes 
in these students. 

"Activities Following the Retreat" 

The participants of the conference reported being significantly more involved in group 
discussions than the control group. Participants also reported reading more journals and 
books than the controls. 

"Participants' Ratings of the Retreat" 

From data received, the indication was that the former drug users were perceived as 
being the most informative discussion leaders. Based on the results of the evaluation, the 
following conclusions were developed: (a) the retreat was particularly effective in increasing 
the participant's level of information regarding drugs; (b) the retreat had a favorable impact 
on the attitudes of undergraduate students, particularly with regard to marihuana; and (c) the 
conference stimulated participants to further acquire and disseminate information related to - 
drug education. 

Swisher, John D., and Horan, John J. "Effective drug attitude change in college students 
via induced cognitive dissonance." Pennsylvania State University, Apr. 6, 1972. 

(The synopsis below is quoted directly from the article.) 

'The 34 male and female undergraduates (predominantly new freshmen) who attended 
a seminar on drug problems during orientation week at The Pennsylvania State University 
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were randomly assigned to an experimental and a control group. The control and experi- 
mental groiips were pre- and post-tested on the 14 item attitude segment of the Drug 
Education Evaluation Scales. 

"A value-preference inventory of the authors' own making was also employed. This 
instrument had a dual function: in theiirst place^ its 18 items yielded a score which sorted 
the sample into two groups, those preferring direct experience and those preferring 
mediated experiences. The instrument also served as a primary component of the experi- 
mental treatment. Upon receiving personal value information, students who prefer direct 
experience could be expected to become dissonant and thus more conservative in their 
views on drugs. Since drugs are really another way of mediating experience, attitudes of 
students in the latter camp ought to remain unaffected. 

''Students who preferred direct experiences, and who were made to feel dissonant about 
holding liberal drug views, showed considerably more conservatism in their attitudes toward 
drugs than similar students who had not yet been exposed to the experimental treatment. 

"On the other hand, students who preferred mediated experiences and who were thus 
not made to feel dissonant during the experimental treatment presented essentially the same 
attitudinal posture as similar students in the control group. (Slight directionality toward ^ 
liberalism in the latter instance should be noted but interpreted with extreme caution.)" 

Swisher, John D.; Warner, Richard W.; and Herr, Edwin L. "Experimental comparison of four 
approaches to drug abuse prevention among njnth and eleventh graders." Journal of 
Counseling Psychology, Vol. 19, No. 4, 1972. Pp. 328-332. 

The purpose of this study was to investigate the effectiveness of four approaches to 
drug abuse prevention among secondary school youth. The four approaches were: 

1. A standard unit in health classes focused on drug abuse. 

2. Relationship counseling groups in which the students were allowed to explore the topic 
of drug abuse in any fashion they chose. The counselor's role was to help the group 
members to accept, understand, and express themselves. The counselor remained 
neutral with regard to the issues and was particularly careful not to impose his opinions 
on the group members. Participating counselors were given training designed to heighten 
their emission of empathy, respect, and genuineness. 

3. Reinforcement counseling groups that included a counselor and two college-age non- 
drug-abusing role models. The role models were selected in advance and given an 
orientation with regard to their role in the group. They facilitated the discussions toward 
reasons for not being involved in drug abuse. The models were selected on the basis that 
although they had not abused drugs, they knew the drug culture. The counselors were 
to keep the discussion focused on alternatives and the use of drugs and to give positive 
reinforcement to statements made by the students which represented behavior and 
attitudes that would result in not becoming involved with drugs at some later time. 

4. Reinforcement counseling groups that had a counselor and two college-age ex-drug- 
abusing role models. The role models were similar to the treatment 3 type models except 
they were chosen on the basis of being former drug abusers. The counselors in this type 
of group functioned in the same manner as the treatment 3 type counselor. 

Evaluation 

Three primary instruments were used to identify and "tap" the three dimensions of the 
problem of drug abuse. The three dimensions were given as cognitive, affective, and 
behavioral. The instruments were: 

1. A 41-item achievement test which measured changes in level of students' knowledge 
regarding the various abuses of drugs. 
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2. A 14-item attitude scale which measured the students' attitudes regarding the various 
issues (e.g., legalization of marihuana). 

3. A 35-item health habits scale which assessed the students' current involvement with 
drugs, their motivations for using drugs, the circumstances in which they abuse drugs, 
and their sources of drugs. 

Conclusion 

The results of the three tests given were: 

1. Both 9th and 11th grades showed an increase in knowledge of drugs. The level of gain in 
drug knowledge was essentially the same in both grades. 

2. There were no significant changes in the students' attitudes toward drugs from pre- to 
posttesting occasions. 

3. There was no significant change in levels of drug use from pre- to posttesting occasions. 

Through the testing of students it was demonstrated that knowledge of drugs can be 
given as easily through the traditional classroom techniques (e.g., health class) as through 
group counseling. They also indicated that none of the four approaches had any impact on 
changing attitudes toward drug abuse or actual use of drugs. 

A second important conclusion resulted from the rating of counselor performance. All 
group sessions were recorded and each of the counselors was rated on the Truax scales in 
terms of criteria assumed to identify relationship counseling and of criteria assumed to 
identify reinforcement counseling. According to the Truax scales of the six counselors, only 
three of them performed above the minimally effective level. The other three were below 
that level. 

Third, the study showed that peer group use was highly correlated with personal use 
of drugs. 

Warner, Richard W., Jr., et al., ''Drug Abuse Prevention: A Behavioral Approach." Pennsyl- 
vania State University, 1972. 

(The first paragraph of the synopsis below is quoted directly from the article.) 

'The participants of the project consisted of 119 ninth-grade students who were cur- 
rently enrolled in a required health class and 3 counselors who were trained in conducting 
the experimental and control-procedures. All students were randomly assigned to one of 
twelve counseling groups. The groups were then randomly assigned to one of four treatment 
conditions. Each counselor (also randomly assigned) conducted one group under each 
experimental condition." 

A 14-item Likert-type scale was administered anonymously to all groups at the beginning 
and end of the 6-week experimental period to measure change in attitude toward the use of 
drugs. The study involved three counseling and one no-treatment control approaches as 
follow: 

A behavioral counseling group with a trained and supervised counselor. The counselor 
was'to have the students consider visible alternates to the use of drugs. 
A cognitive dissonance group with a trained and supervised counselor. This involves 
showing pro-drug attitudes to be inconsistent with other selected values, (e.g. concern 
for ecology). 

A placebo group with a counselor who was instructed to stimulate a discussion of the 
drug problem. The counselor was to listen to and accept the comments of students in a 
nonjudgmental fashion. 

A control group which participated in the regular health unit. 
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Findings: 

The behavioral counseling groups showed the greatest gain in healthy drug attitudes; 
however changes were not significantly greater than those exhibited by the control groups. 
Changes in the behavioral counseling groups (approach 1) were in a healthier direction than 
those displayed by the cognitive dissonance group or the placebo group. 

AN EVALUATION OF THE EFFECT OF A VALUES-ORIENTED DRUG ABUSE EDUCATION 
PROGRAM USING THE RISK-TAKING ATTITUDE QUESTIONNAIRE (1968-1970) 
(ABSTRACT) ^ 

Changes in the behavior of pupils in grades 4-12 of the Coronado Unified School 
District were observed using the Risk-Taking Attitude Questionnaire (RTAQ) to evaluate the 
magnitude and direction of such changes. Experimental groups. in the 12th, 6th, 5th and 4th 
grades were compared to control classes which were not exposed to the drug abuse program. 
Experimental classes were given a values-oriented drug abuse program which contained 
both cognitive and affective components. "Pre" and ''post" administration of the RTAQ on 
the same group was done in most cases; but in some, data contained in previous studies 
were used for comparison. 

The hypotheses of the program were that the amount of drug use and other high-risk 
low-gain behaviors would be less for the experimental groups than for the controls. In addi- 
tion, frequencies of more adjustive behaviors were expected to be higher in the experimental 
groups. A similar pattern of change was expected on the rated risks and gains of behavior. 
The experimental groups should have moved toward more risk and less gain on high-rislK, 
low-gain behaviors; and toward less risk and more gain on low-risk, high-gain behaviors. The 
control groups were expected to move in opposite directions. It was also hypothesized that 
experimental groups would see more effectiveness in actions to control undesirable behavior 
by school, parents, etc., and less effectiveness for actions such as dropping out of school 
than would the control group. 

There was some evidence in favor of every hypothesis with the possible^exception of 
the "action" ratings. Actual frequency of behavior tended to shift in the predicted direction, 
and lower acceptance (and use in the upper age and grade areas) of drugs was clearly 
evident in the experimental groups. Risks and gains attitudes also tended to move into the 
desired pattern. 

The clearest evidence was available for the youngest males and on the gain ratings. 
Frequent differences based on sex were also evident and change points were found at certain 
ages, which indicated that particular types of risk and gain were more relevant at that age. 
The gain of "feeling older" (maturity) seemed important for 12th grade males, while gains of 
"acceptance" (friends) and "thrills" showed more change at other ages. Risk seemed to be 
more relevant for 4th and 6th grade males than for any other groups. 

EVALUATION OF THE 1970-71 CORONADO, CALIFOPtNIA, DRUG ABUSE PREVENTION 
PROGRAM USING THE RISK-TAKING ATTITUDE QUESTIONNAIRE (ABSTRACT) ^ 

This paper examines the pre-post changes which took place during the second full 
year of operation of the Coronado, California, "Values" drug abuse prevention program. 
Experimental and control groups were compared in the fourth, fifth, and sixth grades and 
experimental groups only in the junior and senior high schools. For various reasons, it was 
not possible to carry out the original research design, and comparisons of the experimental- 
control categories were meaningless in the fourth through sixth grades. Analysis of the data 
based on these categories showed little systematic change. 



' This abstract was written by Richard E. Carney, Ph.D., Educators Assistance Institute, Santa Monica, California. 
^This abstract was written by Richard E. Carney, Ph.D., Educators Assistance Institute, Santa Monica, California. 
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VVhen individual classes were ranked on the degree of training for and use of program 
materials and on the amount of "values-orientation" of the teacher, clear indication of 
program effectiveness was found. Males in the most values-oriented classes tended to 
change to lower drug use and greater participation in positive behaviors such as working 
hard; however, girls showed little change. 

Moderate success was obtained in cross-validating prediction equations for behaviors 
using attitude ratings as predictors. Some evidence for a positive effect of the program on 
attitude changes was found at the secondary level, but these^ changes were difficult to 
evaluate because of the lack of control groups. 

When attitude changes were examined in the fourth through sixth grades, some lowered 
predicted involvement by values classes of males in "drug-abuse" behaviors was found. 
Females in the values classes tended to hawe higher predicted scores. Comparisons of 
changes in perceived risk and gain showed strong selective movement by males in values 
classes toward increased ratings of risk and lowered ratings of gain for "drug-abuse" 
behaviors. Females, again, showed an opposite pattern. 

REPORT ON THE USE OF THE RTAVI TO EVALUATE VALUES, ATTITUDES, AND 
BEHAVIORS IN THE MADISON SCHOOL DISTRICT, PHOENIX, ARIZONA (1972) 
(ABSTRACT)'^ 

Groups of 82 fourth grade and 75 seventh grade pupils from Phoenix, Arizona were 
included in a "Valuing" drug-abuse prevention education program during the 1971-72 school 
year. Control groups of 18 fourth grade and 19 seventh grade pupils were used for com- 
parison of pre-post changes that might be attributable to the "Valuing" program. The Risk- 
Taking Attitude-Values Inventory Elementary form was used "pre" and "post" to measure 
five clusters of attitudes and one cluster of 15 reported behaviors. Within each cluster the 
Items were arranged from lowest to highest average and it was hypothesized that an ideal 
outcome would produce progressively greater increases in post averages toward the high 
end of the set of items. That is, the valuing groups should hold constant or decrease low 
frequency (high risk-low gain) behaviors such as drug abuse and increase high frequency 
behaviors such as working and art relative to the control groups. 

Out of 24 possibilities (two groups, two sexes, six clusters), 13 were definitely in favor 
of the ideal outcome, four slightly in favor, six showed no difference and in one case there 
was a result slightly against the valuing program. The most favorable results were obtained 
on the Importance of Values, Ways of Changing Behavior and Frequency of Behavior « 
clusters. Relatively little favorable change was found on the Utility and Expectancy of 
Behavior ratings. 

VALUING AND DRUGS— FINAL 1971-72 REPORT ON THE TEMPE, ARIZONA DRUG 
ABUSE PREVENTION PROGRAM (ABSTRACT)^ 

Pre-post changes in averages on the Risk-Taking Attitude-Values Inventory were exam- 
ined for the fourth, sixth, and eighth grades in the Tempe, Arizona School District #3. 
Various combinations of Valuing (V) and Drug fact (D) programs were compared to control 
groups who received no special attention. Beth changes in reported raw scores and in 
predicted frequencies of behavior were analyzed by ANOVA techniques. Changes in raw 
score Values Importance and Values Nearness ratings were also tested by ANOVA. 

Clearly favorable outcomes in both raw and predicted frequencies of behavior were 
found for the eighth grade males in the V and D + V groups. Drug use was less than 
controls and "good" behaviors such as working and sports were higher.vSimilar but less 
clear-cut findings were obtained for the eighth and sixth grade females in the V group with 

''This abstract was written by Richard E. Carney, Ph.D., Educators Assistance Institute, Santa Monica, California. 
^ This abstract was written by Richard E.Carney, Ph.D., Educators Assistance institute, Santa Monica, California. 
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more negative results in V + D and D groups. Little consistent change was found for any 
group of sixth grade males and for the V + D groups of fourth grade males and females. 

Some'favorable program effect on values ratings was found for every grade and sex 
group. The differences from controls tended to be small and inconsistent from one grade, 
sex, and program group to another. 
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RESOURCES 



The resources listed here provide guidelines for evaluation or give examples of instru- 
ments, measures, or strategies that have been used in evaluation. The reader will have to 
judge for himself the validity of the instruments or measures and their usefulness in evalu- 
ating his program. 

Accountability In Drug Education: A Model for Evaluation, Washington, D.C. The Drug Abuse 
Council, 1973. 

Drug Abuse Research Instrument Inventory. Fourth Edition, Cambridge, Massachusetts. 
Social Systems Analysts. 

Guidelines for Evaluation of Continuing Education Programs in Mental Health, Rockville, 
Maryland: National Clearinghouse for Mental Health Information, National Institute of 
Mental Health. 

Measures Pertaining to Health Education: Drugs; An Annotated Bibliography, Princeton, 
New Jersey: TM Reports No. 9, ERIC Clearinghouse on Tests, Measurements, and 
Evaluation, Educational Testing Service. 

RicharJ->/ Louise G. "Evaluation in Drug Education." School Health Review, Vol. 2, No. 3, 
IRI September 1971, (This article reviews several evaluation studies.) 

Webb, Eugene J.' Campbell, Donald T.; Schwartz, Richard D.; and Sechrest, Lee. Unobtrusive 
Measures: Nonreactive Research in the Social Sciences, Chicago: Rand McNally, 1966. 
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